Skip to content

feat: Add R cataloger#1790

Merged
willmurphyscode merged 8 commits intomainfrom
add-r-cataloger
May 10, 2023
Merged

feat: Add R cataloger#1790
willmurphyscode merged 8 commits intomainfrom
add-r-cataloger

Conversation

@willmurphyscode
Copy link
Copy Markdown
Contributor

Add a cataloger that detects installed R packages by looking for DESCRIPTION files.

fixes: #730

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 5, 2023

Benchmark Test Results

Benchmark results from the latest changes vs base branch
goos: linux%0Agoarch: amd64%0Apkg: github.com/anchore/syft/test/integration%0Acpu: Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz%0A                                                          │ ./.tmp/benchmark-d163d37.txt │%0A                                                          │            sec/op            │%0AImagePackageCatalogers/alpmdb-cataloger-2                                   11.95m ±  1%25%0AImagePackageCatalogers/apkdb-cataloger-2                                    620.0µ ±  1%25%0AImagePackageCatalogers/binary-cataloger-2                                   218.1µ ±  6%25%0AImagePackageCatalogers/dpkgdb-cataloger-2                                   599.7µ ±  4%25%0AImagePackageCatalogers/dotnet-deps-cataloger-2                              1.240m ±  1%25%0AImagePackageCatalogers/go-module-binary-cataloger-2                         95.06µ ±  1%25%0AImagePackageCatalogers/java-cataloger-2                                     13.12m ±  4%25%0AImagePackageCatalogers/graalvm-native-image-cataloger-2                     94.47µ ±  1%25%0AImagePackageCatalogers/javascript-package-cataloger-2                       408.0µ ±  1%25%0AImagePackageCatalogers/nix-store-cataloger-2                                283.9µ ±  2%25%0AImagePackageCatalogers/php-composer-installed-cataloger-2                   780.3µ ± 14%25%0AImagePackageCatalogers/portage-cataloger-2                                  395.2µ ±  2%25%0AImagePackageCatalogers/python-package-cataloger-2                           3.200m ±  2%25%0AImagePackageCatalogers/r-package-cataloger-2                                178.7µ ±  1%25%0AImagePackageCatalogers/rpm-db-cataloger-2                                   534.6µ ±  1%25%0AImagePackageCatalogers/ruby-gemspec-cataloger-2                             910.5µ ±  1%25%0AImagePackageCatalogers/sbom-cataloger-2                                     119.9µ ±  0%25%0Ageomean                                                                     599.9µ%0A%0A                                                          │ ./.tmp/benchmark-d163d37.txt │%0A                                                          │             B/op             │%0AImagePackageCatalogers/alpmdb-cataloger-2                                   5.118Mi ± 0%25%0AImagePackageCatalogers/apkdb-cataloger-2                                    146.3Ki ± 0%25%0AImagePackageCatalogers/binary-cataloger-2                                   31.99Ki ± 0%25%0AImagePackageCatalogers/dpkgdb-cataloger-2                                   170.6Ki ± 0%25%0AImagePackageCatalogers/dotnet-deps-cataloger-2                              410.6Ki ± 0%25%0AImagePackageCatalogers/go-module-binary-cataloger-2                         10.06Ki ± 0%25%0AImagePackageCatalogers/java-cataloger-2                                     2.784Mi ± 0%25%0AImagePackageCatalogers/graalvm-native-image-cataloger-2                     8.750Ki ± 0%25%0AImagePackageCatalogers/javascript-package-cataloger-2                       98.58Ki ± 0%25%0AImagePackageCatalogers/nix-store-cataloger-2                                49.42Ki ± 0%25%0AImagePackageCatalogers/php-composer-installed-cataloger-2                   179.4Ki ± 0%25%0AImagePackageCatalogers/portage-cataloger-2                                  86.40Ki ± 0%25%0AImagePackageCatalogers/python-package-cataloger-2                           977.5Ki ± 0%25%0AImagePackageCatalogers/r-package-cataloger-2                                41.59Ki ± 0%25%0AImagePackageCatalogers/rpm-db-cataloger-2                                   178.2Ki ± 0%25%0AImagePackageCatalogers/ruby-gemspec-cataloger-2                             139.7Ki ± 0%25%0AImagePackageCatalogers/sbom-cataloger-2                                     14.20Ki ± 0%25%0Ageomean                                                                     125.4Ki%0A%0A                                                          │ ./.tmp/benchmark-d163d37.txt │%0A                                                          │          allocs/op           │%0AImagePackageCatalogers/alpmdb-cataloger-2                                    87.63k ± 0%25%0AImagePackageCatalogers/apkdb-cataloger-2                                     3.683k ± 0%25%0AImagePackageCatalogers/binary-cataloger-2                                     896.0 ± 0%25%0AImagePackageCatalogers/dpkgdb-cataloger-2                                    2.997k ± 0%25%0AImagePackageCatalogers/dotnet-deps-cataloger-2                               6.326k ± 0%25%0AImagePackageCatalogers/go-module-binary-cataloger-2                           281.0 ± 0%25%0AImagePackageCatalogers/java-cataloger-2                                      39.47k ± 0%25%0AImagePackageCatalogers/graalvm-native-image-cataloger-2                       228.0 ± 0%25%0AImagePackageCatalogers/javascript-package-cataloger-2                        1.321k ± 0%25%0AImagePackageCatalogers/nix-store-cataloger-2                                  893.0 ± 0%25%0AImagePackageCatalogers/php-composer-installed-cataloger-2                    3.796k ± 0%25%0AImagePackageCatalogers/portage-cataloger-2                                   1.668k ± 0%25%0AImagePackageCatalogers/python-package-cataloger-2                            15.96k ± 0%25%0AImagePackageCatalogers/r-package-cataloger-2                                  805.0 ± 0%25%0AImagePackageCatalogers/rpm-db-cataloger-2                                    3.879k ± 0%25%0AImagePackageCatalogers/ruby-gemspec-cataloger-2                              2.279k ± 0%25%0AImagePackageCatalogers/sbom-cataloger-2                                       394.0 ± 0%25%0Ageomean                                                                      2.467k

Add a cataloger that detects installed R packages by looking for DESCRIPTION
files. The base R package is now picked up in coverageImage tests in
test/cli/packages_cmd_test.go, so increment expected package counts for the
tests that use that image.

Signed-off-by: Will Murphy <will.murphy@anchore.com>
Comment thread syft/pkg/cataloger/r/cataloger_test.go Outdated
Comment thread syft/pkg/cataloger/r/package.go Outdated
Comment thread syft/pkg/cataloger/r/package.go Outdated
Comment thread syft/pkg/cataloger/r/package.go Outdated
Comment thread syft/pkg/cataloger/r/parse_description.go
Comment thread test/integration/catalog_packages_test.go Outdated
Signed-off-by: Will Murphy <will.murphy@anchore.com>
Comment thread syft/pkg/cataloger/r/parse_description.go Outdated
Mostly, don't return packages that have no name or version.

Signed-off-by: Will Murphy <will.murphy@anchore.com>
Signed-off-by: Will Murphy <will.murphy@anchore.com>
Apparently this is set elsewhere.

Signed-off-by: Will Murphy <will.murphy@anchore.com>
Copy link
Copy Markdown
Contributor

@wagoodman wagoodman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice addition! 🎉

Copy link
Copy Markdown
Contributor

@wagoodman wagoodman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realized just after hitting approve that a JSON schema addition needs to be made

@wagoodman
Copy link
Copy Markdown
Contributor

To make a JSON schema update you can add your new metadata struct to https://github.com/anchore/syft/blob/main/schema/json/generate.go#L32 and follow the steps in https://github.com/anchore/syft/blob/main/schema/json/README.md#json-schema on how to bump .

Because the new R package metadata type is a change to the JSON that can
be written, bump the schema.

Signed-off-by: Will Murphy <will.murphy@anchore.com>
Signed-off-by: Will Murphy <will.murphy@anchore.com>
Signed-off-by: Will Murphy <will.murphy@anchore.com>
@willmurphyscode willmurphyscode merged commit da36246 into main May 10, 2023
@willmurphyscode willmurphyscode deleted the add-r-cataloger branch May 10, 2023 16:30
spiffcs added a commit that referenced this pull request May 11, 2023
* main:
  feat: warn if parsing newer SBOM (#1810)
  feat: Add R cataloger (#1790)
  update cosign to v2 release (different go module) (#1805)

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
spiffcs added a commit that referenced this pull request May 18, 2023
* main: (32 commits)
  chore(deps): bump github.com/google/go-containerregistry (#1823)
  chore(deps): bump github.com/sirupsen/logrus from 1.9.0 to 1.9.1 (#1822)
  chore(deps): bump github.com/docker/docker (#1824)
  fix: update field plurality of 8.0.0 schema before release (#1820)
  fix: update cataloger to check for expressions before split (#1819)
  feat: update syft license concept to complex struct (#1743)
  fix: cyclonedx depends-on relationship inverted (#1816)
  fix: retain sbom cataloger relationships (#1509)
  feat: warn if parsing newer SBOM (#1810)
  feat: Add R cataloger (#1790)
  update cosign to v2 release (different go module) (#1805)
  fix: Reduce log spam on unknown relationship type (#1797)
  chore(deps): update bootstrap tools to latest versions (#1807)
  chore(deps): bump golang.org/x/net from 0.9.0 to 0.10.0 (#1802)
  chore(deps): bump github.com/docker/docker (#1795)
  chore(deps): bump github.com/google/go-containerregistry (#1796)
  chore(deps): update bootstrap tools to latest versions (#1792)
  Print package list when extra packages found (#1791)
  chore(deps): update bootstrap tools to latest versions (#1786)
  chore(deps): bump golang.org/x/term from 0.7.0 to 0.8.0 (#1787)
  ...

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
GijsCalis pushed a commit to GijsCalis/syft that referenced this pull request Feb 19, 2024
Add a cataloger that detects installed R packages by looking for DESCRIPTION
files. The base R package is now picked up in coverageImage tests in
test/cli/packages_cmd_test.go, so increment expected package counts for the
tests that use that image.

Signed-off-by: Will Murphy <will.murphy@anchore.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support cataloging R packages

2 participants